Faulty cart wheel detection

ABSTRACT

A system and method of identifying carts exhibiting tendencies that are indicative of damaged or defective wheels. A shopping cart may be identified and tracked visually through one or more surveillance cameras. By comparing the cart&#39;s tracked movement to known symptomatic movement patterns, the system may identify defective or damaged carts. Alternatively, by analyzing movement and positioning of a cart&#39;s swiveling wheels, the system may identify defective or damaged carts. Alternatively, by identifying if a customer has abandoned a cart, the system may identify defective or damaged carts. A notification message may be transmitted to an associate to repair or replace the identified problematic cart. The notification may be displayed on a mobile computing device, a workstation, or other like systems.

RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.13/723,137, filed Dec. 12, 2012, which is expressly incorporated hereinby reference in its entirety.

BACKGROUND

Shopping carts, also known as shopping trolleys in certain countries,are ubiquitous implements in many types of retailers, particularlygrocery stores. Shopping carts are typically provided to customers byretail establishments to increase the customers' convenience andencourage return shopping visits and increased shopping activity. Cartsmay typically have four or more wheels. Many carts have swivel wheels atthe front and non-swiveling rear wheels. Other carts have four swivelingwheels.

Over time and/or due to damage, wheels on carts may become stuck so thatthey exhibit increased spinning or swiveling resistance. Additionally,damaged swiveling wheels may develop wobble, which generally defined assustained oscillation of the swiveling wheels. Such damage or defectsmay increase the amount of effort expended by a person trying to push orpull the cart. A cart with such damage or defects may have a tendency toveer to one side or may not easily turn in one or both directions. Suchproblematic carts may cause user frustration and decrease user comfort,which can discourage customers from returning to the establishment.

What is needed, therefore, is a system for identifying carts thatexhibit symptoms of damaged or defective wheels and for notifying anassociate to remove from circulation and potentially remove or replacethe problematic carts.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the present disclosureare described with reference to the following figures, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified.

FIG. 1 is a schematic block diagram of a faulty wheel detection systemaccording to one embodiment;

FIG. 2 is a flowchart illustrating an exemplary method of detecting afaulty cart wheel; and

FIGS. 3A-3B are overhead illustrations of problematic cart movementpatterns that are symptomatic of damaged or defective wheels.

Corresponding reference characters indicate corresponding componentsthroughout the several views of the drawings. Skilled artisans willappreciate that elements in the figures are illustrated for simplicityand clarity and have not necessarily been drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help to improve understanding of variousembodiments of the present disclosure. Also, common but well-understoodelements that are useful or necessary in a commercially feasibleembodiment are often not depicted in order to facilitate a lessobstructed view of these various embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings that form a part thereof, and in which is shown by way ofillustration specific exemplary embodiments in which the disclosure maybe practiced. These embodiments are described in sufficient detail toenable those skilled in the art to practice the concepts disclosedherein, and it is to be understood that modifications to the variousdisclosed embodiments may be made, and other embodiments may beutilized, without departing from the spirit and scope of the presentdisclosure. The following detailed description is, therefore, not to betaken in a limiting sense.

Reference throughout this specification to “one embodiment,” “anembodiment,” “one example,” or “an example” means that a particularfeature, structure, or characteristic described in connection with theembodiment or example is included in at least one embodiment of thepresent disclosure. Thus, appearances of the phrases “in oneembodiment,” “in an embodiment,” “one example,” or “an example” invarious places throughout this specification are not necessarily allreferring to the same embodiment or example. Furthermore, the particularfeatures, structures, or characteristics may be combined in any suitablecombinations and/or sub-combinations in one or more embodiments orexamples. In addition, it should be appreciated that the figuresprovided herewith are for explanation purposes to persons ordinarilyskilled in the art and that the drawings are not necessarily drawn toscale.

Embodiments in accordance with the present disclosure may be embodied asan apparatus, method, or computer program product. Accordingly, thepresent disclosure may take the form of an entirely hardware-comprisedembodiment, an entirely software-comprised embodiment (includingfirmware, resident software, micro-code, etc.), or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module,” or “system.” Furthermore,embodiments of the present disclosure may take the form of a computerprogram product embodied in any tangible medium of expression havingcomputer-usable program code embodied in the medium.

Any combination of one or more computer-usable or computer-readablemedia may be utilized. For example, a computer-readable medium mayinclude one or more of a portable computer diskette, a hard disk, arandom access memory (RAM) device, a read-only memory (ROM) device, anerasable programmable read-only memory (EPROM or Flash memory) device, aportable compact disc read-only memory (CDROM), an optical storagedevice, and a magnetic storage device. Computer program code forcarrying out operations of the present disclosure may be written in anycombination of one or more programming languages. Such code may becompiled from source code to computer-readable assembly language ormachine code suitable for the device or computer on which the code willbe executed

Embodiments may also be implemented in cloud computing environments. Inthis description and the following claims, “cloud computing” may bedefined as a model for enabling ubiquitous, convenient, on-demandnetwork access to a shared pool of configurable computing resources(e.g., networks, servers, storage, applications, and services) that canbe rapidly provisioned via virtualization and released with minimalmanagement effort or service provider interaction and then scaledaccordingly. A cloud model can be composed of various characteristics(e.g., on-demand self-service, broad network access, resource pooling,rapid elasticity, and measured service), service models (e.g., Softwareas a Service (“SaaS”), Platform as a Service (“PaaS”), andInfrastructure as a Service (“IaaS”)), and deployment models (e.g.,private cloud, community cloud, public cloud, and hybrid cloud).

The flowchart and block diagrams in the attached figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It will also be notedthat each block of the block diagrams and/or flowchart illustrations,and combinations of blocks in the block diagrams and/or flowchartillustrations, may be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions. These computerprogram instructions may also be stored in a computer-readable mediumthat can direct a computer or other programmable data processingapparatus to function in a particular manner, such that the instructionsstored in the computer-readable medium produce an article of manufactureincluding instruction means which implement the function/act specifiedin the flowchart and/or block diagram block or blocks.

Embodiments of the present disclosure comprise methods and systems thatallow retail or other establishments to automatically detect defectiveand/or damaged cart wheels or casters by visually tracking and analyzingmovement patterns followed by the cart. Embodiments include one or morecameras and an image processing module adapted to identify and trackmovement of a cart. If a cart's movement follows a pattern known to besymptomatic of a defective wheel or caster, an alert may be created andtransmitted to an associate to remove the problematic cart from useand/or repair or replace the defective part.

With reference to FIG. 1, faulty wheel detection system 100 comprisescamera 110, image processing module 120, and application server 130. Inembodiments, camera 110 comprises multiple security or surveillancecameras, such as those typically installed on a ceiling or wall atretail, commercial, and industrial establishments. In other embodiments,camera 110 comprises other means capable of capturing still and/or videoimages. In embodiments, camera 110 is adapted to swivel, pan, andotherwise direct its image-capturing apparatus.

Image processing module 120 comprises circuitry, a processor andcomputer-readable instructions in the form of software stored in amemory, combinations thereof, or the like adapted to identify and trackobjects in images captured by camera 110 as described herein. Imageprocessing module 120 is adapted to transmit a signal to applicationserver 130 upon recognition of certain patterns or behaviors in targetobjects. Image processing module 120 can identify objects in the imagescaptured by camera 110 as carts and track each cart's location as itmoves within the view of one or more cameras 110. Image processingmodule 120 may include known visual recognition and image processingsoftware and/or hardware techniques to distinguish between carts,people, and other objects in each image. Image processing module 120 canlink sequential images from camera 110 and track an identified cart'smovement through the sequence.

In embodiments, image processing module 120 can detect what type ofdamage or defect is afflicting a cart. For example, if a cart exhibits atendency to pull to the left, a problem with the left front wheel may beindicated. If certain movement patterns are symptomatic of a front wheelthat has an increased resistance to swiveling, such a problem can beidentified by the image processing module 120.

Application server 130 comprises circuitry, a processor andcomputer-readable instructions in the form of software stored in amemory, combinations thereof, or the like adapted to receive a signalfrom image processing module 120 regarding image analysis and transmitan alert to an associate smartphone application 140. Data transmitted byapplication server 140 may include data regarding movement patternstracked by image processing module 120. For example, data transmitted byapplication server 130 may include a last-known location of a cart.Additional data may include data related to the type of damage or defectdetected, such as which wheel may be damaged and possible correctivemeasures.

Application server 130 is adapted to receive data from image processingmodule 120 and create an alert to transmit to associate smartphone app140 over network 150. An alert may include a last-known location of theidentified cart, details of the cart, an image of the customer using thecart, and a suggested course of action to remove the cart from useand/or repair the cart. An alert may be presented as a notificationwithin a graphical user interface of a smartphone app 140 used generallyfor management functions within a retail store or other establishment.In alternative embodiments, an alert is transmitted to a workstation,for example an onsite, offsite, and/or back office computer workstation.In other embodiments, an alert is transmitted to a mobile device. Amobile device may comprise a smartphone, a personal digital assistant,Wi-Fi mobile device, or other type of mobile computing device.

Smartphone app 140 comprises software, hardware, and/or memory in amobile computing device in communication with application server 130 vianetwork 150. Smartphone app 140 is adapted to display an alert to anassociate in response to receiving the alert from application server140. In embodiments, an associate may request additional informationthrough smartphone app 140. In alternative embodiments, network 150comprises any communication network including, but not limited to: awireless network, a cellular network, an intranet, the Internet, orcombinations thereof.

In operation, faulty wheel detection system 100 is adapted to capturevideo or image sequences, identify a cart appearing in the video orimages, track the cart's movement and/or related behavior, identifycarts having faulty, defective, or damaged wheels or casters, and notifyan associate for corrective action. Referring now to FIG. 2, embodimentsof the present disclosure comprise method 200. At operation 210, camera110 captures video or a sequence of images depicting a cart. Inembodiments, the cart is a grocery cart pushed by a customer whileshopping in a grocery store.

At operation 220, image processing module 120 receives images fromcamera 110 and identifies if any carts are depicted therein. Atoperation 230, image processing module 120 tracks the movement of anyidentified carts or like objects. In embodiments, image processingmodule 120 is adapted to identify the orientation of each identifiedcart (i.e., which end of the cart is the front and which is the rear).Tracking cart movement may be accomplished over multiple cameras 110feeding images into image processing module 120. Coordination betweenmultiple cameras 110 may be managed by image processing module 120 tofollow an object as it passes between zones covered by respectivecameras 110. Image processing module 120 may transmit instructions tocameras 110 to aim at specific coordinates, zoom level, and the like tooptimize tracking of carts and other objects.

At operation 240, image processing module 120 compares a cart's trackedmovement to known patterns that are symptomatic of defective or damagedwheels. Such movement patterns may typically be caused by a defectivewheel, which may cause a cart to favor one direction, exhibit unsteadybehavior, or otherwise follow patterns indicative of damaged ordefective cart wheels. Image processing module 120 can compare observedmovement patterns against known movement patterns that are symptomaticof damaged or defective cart wheels. In embodiments, the observedpattern is assigned a match score based on how similar it is to a knownsymptomatic movement pattern. In an embodiment, a score threshold isimplemented, such that if an observed movement pattern receives a matchscore above the threshold, the observed movement is deemed a match andindicative of a damaged or defective cart wheel.

In embodiments, image processing module 120 tracks a cart's path bylocating an approximate visual centroid of the cart and tracking themovement of the centroid over time. Alternatively, image processingmodule 120 may track both the centroid and the cart's facing direction.In alternative embodiments, image processing module 120 identifies andtracks the cart's edges and/or corners.

Upon identifying a cart with one or more suspected defective wheels,image processing module 120 transmits a signal to application server 130regarding the identified problematic cart. In embodiments, thetransmitted signal includes identifying information for the cart and/orperson using the cart. For example, image processing module 120 maytransmit an image depicting the cart and/or the customer using the cartto application server 130.

At operation 250, application server 130 creates an alert correspondingto the identified shopping cart and transmits the alert to an associate.The alert may instruct the associate to remove the cart from use and/orplace the cart in a repair queue. In an embodiment, the alert includesan image of a grocery store customer pushing the identified defectivecart and/or the cart's last known location so that an associate can findthe cart in the store and trade the cart out for a functional cart. Inanother embodiment, an alert is transmitted to a cashier upon thecustomer arriving at the cashier so that the cashier can compensate thecustomer for the inconvenience of the defective shopping cart.

After identifying and creating an alert for a problematic cart,corrective action may be effected on the identified cart. Inembodiments, an associate removes the cart having a suspected defectivewheel from active use so that another customer will not receive and usethat cart. In alternative embodiments, an order to repair the identifiedcart may be issued.

Referring now to FIG. 3A, various movement patterns by a cart 300 mayevince a defective wheel. For example, a cart 300 with a defective wheelmay veer in one direction or favor that direction. Image processingmodule 120 is adapted to track a path 310A followed by cart 300. Acustomer pushing the cart 300 may occasionally notice that the cart 300has not maintained the desired direction and may occasionally make acorresponding course correction 320A. Such a course correction 320A mayindicate to image processing module 120 that the cart 300 has adefective or damaged wheel. Alternatively, image processing module 120can detect if a cart 300 persistently veers to one side and thereforeascertain that the cart 300 may have a defective wheel.

Referring now to FIG. 3B, a defective or damaged wheel may increase thecornering difficulty of the cart 300. Thus, a problematic cart may beone that has a wider turning radius than is typical for that modeland/or size of cart. Accordingly, image processing module 120 may trackpath 310B of a cart 300 and analyze the cart's 300 turning radius as thecustomer pushes it around corners. A customer may make a wider turn thanis typical, followed by a corresponding course correction as depicted at320B. Such wide turns and corrections 320B may indicate to imageprocessing module 120 that the cart 300 has a defective or damagedwheel. Alternatively, if the cart 300 exhibits a tendency toconsistently make wider turns when turning toward one direction thanwhen turning the other, image processing module 120 may ascertain thatthe cart 300 has a damaged or defective wheel.

Course deviations and corresponding corrections may be more likely tooccur during an early phase of a shopping trip before the customer hasacclimated to the problematic cart's idiosyncrasies. Accordingly,embodiments of the present disclosure comprise an image processingmodule 120 adapted to particularly focus on a shopping cart's movementpatterns soon after a customer begins using that shopping cart.

Cart abandonment by a customer during a shopping trip may indicate thatthe cart has a damaged or defective wheel. A customer may begin using acart shortly before or after entering a store and soon thereafterdiscover that the cart is damaged or defective. Upon realizing that thecart is problematic, the customer may leave the cart and retrieveanother cart or complete the shopping trip without a cart. An abandonedcart may remain where the customer left it until a store associateretrieves it or another customer begins using it. Until then, anabandoned cart could pose a hazard or nuisance to other customers. If asecond customer finds and begins using an abandoned problematic cart,the second customer may experience the same negative effects from thedamaged or defective cart. Accordingly, in embodiments, image processingmodule 120 is adapted to identify if a cart has been abandoned. Imageprocessing module 120 can identify a person pushing a cart and associatethat person with the cart. If the person leaves the cart for a certainamount of time or moves a certain distance from the cart, imageprocessing module 120 may ascertain that the person has abandoned thecart. The specific time and distance thresholds may be dependent onlocal factors, such as the type of retail establishment and size of thebuilding. The time and distance thresholds may be sufficiently large toaccount for customers that leave a cart while searching out items butwith the intent to continue to use the cart.

Alternatively, if the person begins using a second cart after leavingthe first cart, image processing module 120 may ascertain that theperson has abandoned the first cart. In other embodiments, imageprocessing module 120 can use image recognition techniques to identifyif any objects are in a cart. If a customer leaves such objects in acart, then it may be presumed that the customer intends to return to thecart to continue shopping with it. Conversely, if a customer removes allor most items from a cart, then the cart may be identified as abandoned.

Embodiments of the present disclosure analyze the position of a cart'sswiveling wheels to determine if the wheels are damaged or defective. Inone embodiment, image processing module 120 analyzes captured images orvideo to determine the orientation of the swiveling wheels in relationto each other. Image processing module 120 may further analyze theswiveling wheels' orientation in relation to the cart's direction. Ifthe swiveling wheels do not maintain an expected level of coordinationto each other and/or the cart, image processing module 120 may ascertainthat the cart 300 has a damaged or defective wheel.

Embodiments of the present disclosure are adapted to analyze carts toidentify sustained oscillation of the swiveling wheels (also known aswheel wobble). In an embodiment, image processing module 120 analyzes avideo feed of the cart to determine a wheel's oscillation frequency. Ifthe observed frequency is over a predetermined threshold, an alert maybe created to replace and/or repair the cart wheel. Additionalparameters could include oscillation duration; for example: an alert iscreated if the swiveling wheel exhibits sustained wobble for at leastfour seconds. Alternative thresholds may be selected as desired. Wheelwobble may occur at specific frequencies that are dependent, among otherthings, on the cart's characteristics and forward velocity. Accordingly,in an embodiment, image processing module 120 determines the cart'svelocity and calculates an expected wheel wobble frequency range for thecart for that velocity. Image processing module 120 may then analyze avideo feed of the cart to determine the wheels' oscillation frequencyand create an alert if appropriate.

Although the present disclosure is described in terms of certainpreferred embodiments, other embodiments will be apparent to those ofordinary skill in the art, given the benefit of this disclosure,including embodiments that do not provide all of the benefits andfeatures set forth herein, which are also within the scope of thisdisclosure. It is to be understood that other embodiments may beutilized, without departing from the spirit and scope of the presentdisclosure.

What is claimed is:
 1. A computer-implemented method of detecting afaulty cart wheel, comprising: receiving, at an image processing module,an image sequence depicting a cart; identifying, by the image processingmodule, a swiveling wheel of the cart; determining, by the imageprocessing module, an observed oscillation frequency of the swivelingwheel; if the observed oscillation frequency exceeds an oscillationfrequency threshold, transmitting, by an application server to a usercomputing device, an alert regarding the cart.